{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Pytorch中分布式的基本使用流程如下:\n",
    "* 1.使用distributed包的其他函数之前，使用`init_process_group`初始化进程组，同事初始化distributed包\n",
    "* 2.如果需要进行小组内通信，使用`new_group`创建分组\n",
    "* 3.创建数据并行模型`DDP(model,device_id=device_ids)`\n",
    "* 4.为数据集创建`Sample`\n",
    "* 5.使用启动工具torch.distributed.launch在每个主机上执行一次脚本，开始训练\n",
    "* 6.使用`destory_process_group()`销毁进程组\n",
    "#### 下面有使用模板"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}